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STABLE ADIABATIC TIMES FOR A CONTINUOUS 
EVOLUTION OF MARKOV CHAINS 


By Kyle Bradford 
University of Nevada, Reno 

This paper continues the discussion on the stability of time- 
inhomogeneous Markov chains. In particular, this paper defines a 
time-inhomogeneous, discrete-time Markov chain governed by a con¬ 
tinuous evolution in the appropriate martrix space. This matrix space, 
, is the space of all stochastic matrices that are irreducible and 
aperiodic. For this new type of evolution there is a definition of a spe¬ 
cific type of stability called the stable adiabatic time. This measure is 
bounded by a function of the optimal mixing time over the evolution. 
Namely, for a time-inhomogeneous, discrete-time Markov chain gov¬ 
erned by a continuous evolution through a function P : [0,1] 
and 0 < e < 

2y/n 

tsadCP, e) < 

where L is a Lipschitz constant related to the function P. 
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— 2y/nt)t 
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1 . Introduction. The stability of Markov chains is relevant to many 
applications in math and science. For background literature on Markov 
chains one can reference [8, 9, 13]. The first type of stability often encoun¬ 
tered in a discussion of time-homogeneous, irreducible and aperiodic Markov 
chains is the mixing time [1, 13]. || • \\tv is reserved as the total variation 
norm and || • ||fc as the norm. 

Definition 1. For e > 0 the mixing time of a time-homogeneous, ir¬ 
reducible and aperiodie Markov chain governed by a probability transition 
matrix P, which has unique stationary distribution n, is defined as: 

(1) tjnixfP, e) = inf{r E N : ||z/P^ - tt\\tv < e} 

over all distributions v. 

There are many examples of applications of the mixing time [15]. Although 
time-homogeneous Markov chains have been thoroughly studied, the stabil¬ 
ity of time-inhomogeneous Markov chains is much less attainable. People 




have attempted to discuss a related mixing time for time-inhomogeneous 
Markov chains [16, 17, 18]. Although these attempts have been for gen¬ 
eral time-inhomogeneous Markov chains, some time-inhomogeneous Markov 
chains with many real world applications have been studied. The types of 
time-inhomogeneous Markov chains discussed in this paper are best de¬ 
scribed as adiabatic. 

This article continues the effort in [3, 4, 11] to bound the stable adia¬ 
batic time of an evolving, time-inhomogeneous Markov chain by a function 
of the largest mixing time over the entire evolution. Specifically this pa¬ 
per makes three important contributions: 1) finding an exact bound rather 
than an asymptotic bound, 2) finding a tighter, optimal bound of the stable 
adiabatic time and 3) expanding the types of evolutions to include all con¬ 
tinuous transitions in the appropriate matrix space. Some of the strongest 
applications of the adiabatic time and the stable adiabatic time come from 
quantum physics and quantum computation. Namely, the quantum adia¬ 
batic theorem from physics [7, 10] and quantum adiabatic computing [12]. 
There is a strong presence of adiabatic processes in optimization algorithms 
in queueing systems [6], network design [14] and network performance [19]. 
There is also an application to the stability of an Ising model with Glauber 
dynamics [3] . Many of these applications were discussed in detail in previous 
works. For example, the quantum adiabatic theorem was discussed in detail 
in [2, 3, 4, 11] and the quantum computation applications were discussed in 

[ 4 ]. 

In [4] the time-inhomogeneous Markov chain was specifically governed 
by a convex-combination evolution of two irreducible, aperiodic probability 
transition matrices. In particular there were matrices Pq and Pi and P^ = 

(1 — t)Po -|-tPi. Given a large integer T the probability transition matrix at 
time k <T for the time-inhomogeneous Markov chain was P^. Naturally if 

T 

stochastic matrices Pq and Pi are both irreducible and aperiodic, then Pg 
is both irreducible and aperiodic for s E [0,1]. This allows for a definition 
of the mixing time for each s € [0,1]. Taking the supremum of all of these 
mixing times is one of the ways that one can discuss stability for the time- 
inhomogeneous Markov chains with probability transition matrices P k . The 

T 

following definition makes this formal. 

Definition 2. Fore > 0 the largest mixing time of a time-inhomogeneous, 
discrete-time Markov chain governed by a cony ex-combination evolution be¬ 
tween the irreducible and aperiodic Pq and Pi 

tmix (P0 ? Pi; e) — sup {tmia;(Ps) c)}- 

se[o,i] 


( 2 ) 




This paper has already mentioned the stable adiabatic time a few times 
without giving the formal definition. Now there is enough background infor¬ 
mation to make this definition for convex-combination evolutions. This was 
the main object of study in [4] and will motivate the analogue that we will 
use in this paper. 

Definition 3. Fore > 0 the stable adiabatic time of a time-inhomogeneous, 
discrete-time Markov chain governed by a convex-combination evolution be¬ 
tween the irreducible and aperiodic Pq and Pi, which has unique stationary 
distribution for the probability transition matrix Pjc, is defined as : 

T T 

( 3 ) 

isad(Po,Pi,e) = mf{r e N : HvroPi • • • Pk - vik ||ri/ < e for 1 < k <T}. 

T T T 

The stable adiabatic time is another type of stability for these types of 
time-inhomogeneous Markov chains. It is natural to ask how the two previ¬ 
ous definitions compare. This was discussed in [4] for these specific convex- 
combination evolutions. The following asymptotic result was discovered in 

[4] relating the stable adiabatic time and the largest mixing time. 

Theorem 1. Given a time-inhomogeneous, discrete-time Markov chain 
governed by a convex-combination evolution between the irreducible and ape¬ 
riodic Pq and Pi, for any e > 0, 


( 4 ) 


tsad(Po, Pi, e) = O 


Pi, c/2) 


The main goal of this paper is to expand the types of evolutions that can 
take place. To elaborate, first let A4n([0,1]) be the collection of all n x n 
matrices with entries in [0,1]. Define Vn = {P € A4n([0,1]) : PI = 1} where 
1 is the n dimensional column vector with all entries 1 and define 


= {P G Pn : P is irreducible and aperiodic}. 

To describe continuity in this matrix space the standard matrix norm will 
be used. Specifically for a matrix M the matrix norm is defined as ||M|| = 
maXi, IIi^M111 where the maximum is taken over all probability distributions 
u. This paper considers continuous functions P : [0,1] —)• P^“ with respect 
to the Matrix norm to build the more general types of evolutions. One 
can now allow the time-inhomogeneous Markov chains to be governed by a 
continuous evolution defined through the function P. Given a large integer T 
the probability transition matrix at time k <T for the time-inhomogeneous 





Markov chain was P(^). Because all probability transition matrices are 
in a mixing time exists for all s G [0,1]. The supremum can be taken 
again to make a metric for stability for time-inhomogeneous Markov chains 
governed by these continuous evolutions. Note the difference between this 
definition and Definition 2. 


Definition 4. For e > 0 the largest mixing time of a time-inhomogeneous, 
discrete-time Markov chain governed by a continuous evolution in V^, writ¬ 
ten as tmix{Poo,^), is defined as follows: 

(5) tmix (P oo; e) — sup e)}. 

se[o,i] 

Finally the version of the stable adiabatic time used in this paper can be 
introduced. The key difference we Definition 3 is the type of evolution. This 
version of the stable adiabatic time allows for a more general, continuous 
evolution in Vff- 


Definition 5. Fore > 0 the stable adiabatic time of a time-inhomogeneous, 
discrete-time Markov chain governed by a continuous evolution in , writ¬ 
ten as tsarf(P,e), is defined as follows: 

( 6 ) 

4,d(P,e) =inf|rGN: 

for 1 < k < T}. 



With all of these definitions formally laid out it can be said that the pur¬ 
pose of this paper is to find a relationship between tsad(P, e) and tmixfPoo, e) 
in an analogous way as Theorem 1. The machinery in this paper allows for 
a better result and derives an optimal result. The rest of the paper is orga¬ 
nized as follows: Section 2 introduces the necessary background information 
to allow for a succinct proof of the main result, Section 3 gives the main 
result of the paper and gives a detailed proof of the main result. Section 4 
gives a context of the importance of the result and additional proofs and 
argumentation is outlined in Section 5. 


2 . Supporting Material . Whenever one wants to make a proof 
about continuous function spaces, a common proof technique involves us¬ 
ing a dense subset known as the Lipschitz continuous function with finite 
Lipschitz constant. This section introduces two important propositions that 










aide the proof of the main result. For both the first proposition and the 
main result in Section 3 using Lipschitz continuous functions allows for a 
keen insight as to what commands these time-inhomogeneous Markov chains 
governed by a continuous evolution. In this matrix space the following defi¬ 
nition of a Lipschitz continuous function is used. 

Definition 6. A function P* : [0,1] —^ Adn([0,1]) is Lipschitz if there 
exists a positive constant L, called the Lipschitz constant, so that 

(7) ||P*(x)-P*(y)|| <L|x-y| 

for x,y G [0,1]. 

The function P : [0,1] —>■ creates a function vr : [0, 1] —)• M"". By 

definition P is continuous with respect to the matrix norm, so a natural 
question is whether vr is a continuous function with respect to the total 
variation norm. The following proposition declares that it is. This in and 
of itself is not that surprising, but the nature of how it is continuous gives 
information that will be necessary in proving the main result. 

Proposition 1. Let a = infse[o,i]{o'('S)} where a{s) is the smallest 
nonzero singular value o/I — P(s). 

If P : [0,1] —)• Vlf is a continuous function with respect to the matrix norm, 
then vr : [0,1] M"" is uniformly continuous with respect to the total vari¬ 

ation norm. In particular, for e > 0 there exists a positive constant L such 
that for s G [0,1] and 


SLn^^ ’ 

t G {[0,1] : |f — s| < 5} implies that ||7r(f) — 7r(s)||Ti/ < e- 

Notice that in the above proposition the continuity depends on the small¬ 
est nonzero singular value of the function P throughout the entire evolu¬ 
tion. This value cr has information relating to the largest mixing time of P 
throughout the entire evolution. The following proposition makes this point. 

Proposition 2. Let P : [0,1] —)• Vff he a continuous function with re¬ 
spect to the matrix norm. Let a = infse[o,i]{<^('S)} where a{s) is the smallest 
nonzero singular value o/I — P(s). 





Given e > 0 


(9) <t™x(Poo,e). 

a 

Instead of including a proof of Proposition 2 note that the proof falls 
rather directly from a similar argument in [4]. In this paper one can find a 
similar relationship between the smallest nonzero singular value of a matrix 
and its mixing time. Here the only thing to note is that the mixing time 
of time-homogeneous Markov chain associated with the smallest nonzero 
singular value is smaller that the supremum of all mixing times throughout 
the entire evolution. 

This provides all the necessary background to approach our main result. 
This result is now addressed in Section 3. 


3 . Main Result . The main result of this paper is given in the following 
theorem and proven in this section. It will provide the necessary analogue 
for the bound on the stable adiabatic time for time-inhomogeneous Markov 
chains governed by a continuous evolution by a function of the largest mixing 
time over the entire evolution. Note that this result differs from Theorem 1 
by not being an asymptotic result and having a lower power of the largest 
mixing time bound the stable adiabatic time. After this theorem is proven 
the impact of the result will be discussed in Section 4. 

Theorem 2. Given a time-inhomogeneous, discrete-time Markov chain 
governed by a continuous evolution in V^, for 0 < e < and P : [0,1] —)• 
a continuous function with respect to the matrix norm we have that 

(10) t^adiP, e) < 3^'^^L.(Poo,6) 

(I - 2y/ne)e 

Proof. 

For the proof first let 0 < e < and let P : [0,1] —)• be a continuous 

function with respect to the matrix norm. It is important to remember that 
^mia;(PooT/2) exists and is a natural number. 


Recall that the space of Lipschitz continuous functions from [0,1] to 
with finite Lipschitz constant is dense in the space of continuous functions 
from [0,1] to V^. This implies that one can find a Lipschitz continuous 
function P* : [0,1] —)• with Lipschitz constant L such that 


|P(i)-P*(t)|| < 


‘^tmix (P oo) c/2) 





for all t G [0,1]. 

The goal of this proof is to select a value of T large enough so that 



for 1 < /c < T. 

Let 

^ 3n3/2Lt^,^(P^,e/2) ^ 

(1 — 2-yne)e 

At this point the proof is decomposed into two parts. 
Part 1. Assume that k > tmix(Pooj e/2) 

Let = A: - (Poo, e/2). 


Observe that 



where un = vr(0)P ( i ) P (|) • • • P (f), P? = P (|) • • • P (|) . 

By continuing this process for P (y) for i > + 2, it can be shown that 















By the triangle inequality, it can be shown that 



Because 2\\^ — v\\tv = whenever /r and u \s a probability distribu¬ 

tion, 


7r(0)P 
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TV 






TV 


£=0 


where = un (P (|)) (P - P (|)) . 

Notice that for 0 < i < k — N — 2, P^_|_ 2 _|_£ is a probability transition matrix. 
This will imply that 


|,,o-po II 
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Therefore 



It is clear that (P {^)Y is a probability vector for 0 < £ < A: — iV — 2, so 
naturally 


-(0)P(|)p(| 


< max 




TV 




k-N 


k 

— TT I — 
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k-N-2 


n E 


max 


1=0 


V P 


N + l + £ 


TV 

-P.A 


where the maximum is taken over all probability vectors u. 


Because k — N = tmix(Poo, e/2) > tmix(P {^) , e/2), it is easy to see that 



Observe that the terms in the sum of the right hand side of the inequality 
are now the matrix norms for the matrices P _ p for 0 < £ < 

k — N — 2. This would imply that 
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By adding and subtracting the same value to the above inequality and then 
using the triangle inequality 


vr(0)P - P - •••? 


k-l 


P ^ -vr ^ 


TV 


k-N-2 


< - + - V 
“2 2 ^ 

k-N- 

£=0 
k-N- 


N + l + £ 


N + l + £ 
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N + l + £ 


£=0 


P* - - P - 


Using the density of the Lipschitz continuous functions with finite Lipschitz 
constant in the continuous function space 


7r(0)P 
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k-l 


, k-N-2 

“2 2 ^ 
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k-N-2 
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TV 
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+ E 
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^tmix (P cxD) f/2) 


Because P* : [0,1] —>• is a Lipschitz continuous function with Lipschitz 

constant L, it can be shown that 


7r(0)P 


k-l 


— vr 


TV 


- k-N-2 
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After relabeling the sum 


7r(0)P 


k-l 


— vr 


<\ + ^{k-N-l){k-N) 


+ 


^tmix (PcxD) f^/2) 

Because k - N = tmix(Poo, e/2) 


(k-N-l). 


7r(0)P 


k-l 


— vr 


“ T (Poo, e/2). 


T was selected to be large enough. In fact, 


rp _ (Pcxo, e/2) ^ 

(1 — 2^/ne)e ~ e 


TV 


TV 


Finally it is shown that 



Part 2. Assume that k < tmix (Poo, e/2) 


First notice that 



Repeating this process, it can be shown that 

























By the triangle inequality 


vr(0)P - P - •••? 
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Because P^ is a probability transition matrix 
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Using Proposition 1 it is clear that as long as 

3Ln^/'^tmix{'Poo,e/2) 


T > 


ea 


one has that 


TT 


j - 1 


— TT — 
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< 


This would imply that 


7r(0)P - P - 
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TV 


k-l 
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ke 


^mix(Poo) ^/2) 


P - -vr - 


TV 


^mix(Poo) f/2) 
< e. 


Proposition 2 implies that 


^ ^ 3Ln3/2t^.^(P^,e/2) ^ 3Ln^/H 

mix (Poo ? 6/2) 


(1 - 2v^e)e 
This completes our proof. 
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4. Conclusion. Notice that an immediate consequence of Theorem 2 
is that there is a tighter asymptotic bound when compared to the previous 
result in Theorem 1. Also convex-combination evolutions are a specific type 
of continuous evolution, so the class of evolutions is much broader. The 
following corollary sums up these two points. 

Corollary 1. Given a time-inhomogeneous, discrete-time Markov chain 
governed by a continuous evolution in V^, for e > 0 and P : [0,1] ^ a 
continuous function with respect to the matrix norm we have that 

(11) t,ad{P, e) = 0 . 

A hnal question that one might have is whether this bound is optimal 
and the answer is that it is optimal. To show this it suffices to find one 
specific function P such that the stable adiabatic time is exactly a constant 
multiplied my the square of the largest mixing time divided by e. For this 


one can consider 

a convex-combination evolution. Here let 
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As shown in [3] the adiabatic time for this convex-combination evolution 
is of asymptotic order of the square of the largest mixing time divided by e. 
The only inequality that must hold for the adiabatic time, rather than the 
stable adiabatic time, is for HvroPi • • • Px — t^t\\tv < Naturally, for all 

T T T 

the other inequalities to hold HttoP^ • • • P ic — TTk \\tv < e where 1 < A; < T 

T T T 

one must select a value of T at least as large as a constant multiplied by 
the square of the largest mixing time divided by e. The result in this paper, 
however, guarantees that this value of T must be of the same asymptotic 
order. 

This shows that the result from Corollary 1 is optimal. 


5. Proofs. 







5.1. Proof of Proposition 1. 

To begin, consider the creation of an orthonormal basis of eigenvectors as¬ 
sociated with (I — P(s))(I — P(s))^ with respect to || • II 2 throngh a singular 
value decomposition of (I — P(s)), where s G [0,1]. 

Here let cri(s) > • • • > an-i{s) = a{s) be the positive singular values of (I — 
P(s)) with respect to the Euclidean inner product. This implies that there 
exists an orthonormal basis {vi(s), • • • , Vn(s)} such that Vj(s)(I —P(s))(I — 
P(s))'^ = (T|(s)vj(s) for 1 < j < n — 1 and Vn(s)(I — P(s))(I — P(s))^ = 0 . 

Here Vn(s) = 7 r(s)/|| 7 r(s)|| 2 . 

To show continuity at s let e > 0 and first notice that for any t G [0,1], 
( 7 r(t) - 7 r(s))(I[ - P(s)) = 7 r(t)(P(t) - P(s)). 

Using the Euclidean norm, it can easily be seen that if P(t) 7 ^ P(s) and 
t s, then 

||( 7 r(t) - 7 r(s))(I-P(s ))||2 ^ IK(t)(P(t) -P(s ))||2 
||7r(t)- 7 r(s )||2 ||7r(t)- 7 r(s )||2 

Throughout this proof < •, • > we denote the Euclidean inner product. 

For 1 < j < n let Cj{s,t) =< 7r(t) — 7r(s),Vj(s) >. Then 7 r(t) — tt{s) = 
Ej=iCj(s,t)vj(s). 




This will imply that 


(vr(t)-7r(s))(II-P(s))||i 
lk(t) -vr(s)||| 

^ - 7r(.s))(I - P(.s)), (7r(t) - 7r(s))(I - P(s)) > 

< 7r(t) — 7r(s), 7r(t) — 7r(s) > 

^ < 7r(t) - 7r(s), (7r(t) - 7r(s))(I - P(s))(I - P(g))'^ > 

< 7r(t) — 7r(s), 7r(t) — 7r(s) > 

< E?=iCi(s,t)vj(s),X^”^iCj(s,t)vj(s) > 


EU^]{s)c]is,t) 


EUcj{s,t) 


= crl_^{s) 1 - 


= fTLl(s) 1 - 


clis,t) 




< 7r(t) - 7r(s), Vn(s) >' 


||7r(t)-7r(s)||2 

Letting w(s,t) = (7r(t) — 7r(s))/||7r(t) — 7r(s)||2, it can be shown that 


o-2_i(s) (l - (< w(s,t), Vn(s) < 


|7r(t)(P(t) -P(g) 
lk(i) -vr(s)||i 

Because w(s,t) and Vn(s) are unit vectors, the fact that 

l|w(s,t)||i - 2 < ’w(s,t),Vn(s) > +||Vn(s)||| = ||’w(s,t) - Vn(s 
can be used to show that 

1 , 


1- < w{s,t),Vnis) >= ^||w(s,t) - Vn(s)||| 


and the fact that 


|w(s,t)||i + 2 < w(s,t),Vn(s) > +||Vn(s)||i = ||w(s, t) + Vn(s 


can be used to show that 











From this it is clear that 1 — (< w(s,t), Vn(s) >)^ = ||w(s,t) — Vn(s )||2 • 
||w(s,t) + Vn(s)||2/4. Plugging this into the previous equation 




w{s,t) - Vn{s)\\l ■ ||w(s,t) + Vn(s)||| < 


|7r(t)(P(t)-P(s))||i 


4 .. Ht)-7r(s)||i 

After performing some basic algebra 

2||7r(t)(P(t)-P(s))||2 


K(t) -vr(s)||2 < 


CJn-l(s)||w(s,t) - Vn(s )||2 • ||w(s,t) + Vn(s)||2' 


Notice that < w(s, t), 1 > /^/n = 0 and < Vn(s), 1 > /y/n = 1/ (-\/n||7r(s)||2) 
for all t G [0,1]. Because these are the scalar components of the projections 
of w(s,t) and Vn(s) onto 1 respectively,it can be shown that the minimum 
possible value for ||w(s,t) — Vn(s )||2 and ||w(s,t) + Vn(s )||2 is at least 1/ 
(v^||7r(s)||2). 


This shows that 


K(i) -vr(s)||2 < 


< 


2n||7r(s)||| • ||7r(t)(P(f) - P(s ))||2 

O-n-l(s) 

2n||7r(t)(P(f) - P(s ))||2 

O-n-l(s) 

2n||7r(t)(P(f) - P(s ))||2 
cj(s) 


Let a = min^g[o,i]{cr(s)}. 

Again for x, y G M”’ such that x and y are probability measures, it is un¬ 
derstood that 


1 

2 


X 


- yh < 


I II ^ V^ll II 

|x - y\\TV < —\\^ - ylb- 


This will imply that 


7r(t) 


- vr s 


n3/2||vr(t)(P(t)-P(s))||i 

a 

maXi, ||i/(P(t) — P(s))||i 


cr 

where the maximum is taken over all vectors i' such that ||i^||i = 1. 










Using the matrix norm notation one can conclude that 


TT{t) 


7r(s)||ri/ < 


n^/^||P(t) 

a 


P(^)ll 


Notice that the space of Lipschitz continuous functions mapping [0,1] to 
are dense in the space of continuous functions mapping [0,1] to V^. This 
implies that there exists a Lipschitz function P* with Lipschitz constant L 
such that 


Pit) 


P*(t)|| < 


ae 

3n3/2 


for all t G [0,1]. 


One can use the triangle inequality along with this density argument to 
conclude that 


K(t) - vr(s)||T\/ < 


n 


3/2||p(t) _p(. 


cr 


n 


3/2 


n 


a 

3/2 


|P*(t) - P*(s) + P(t) - P*it) + P*(s) - P(s) 


< (||P*(t) - P*(S)|| + ||P(t) - P*(t)|| + ||P*(S) - P(s) 

_ n3/2||P*(t)-P*(s)|| 2e 

“ a ^ y 

Because P* is Lipschitz continuous with Lipschitz constant L, one has that 
||P*(t) — P*(s)|| < Lit — s\ for all t,s € [0,1]. 


This shows that 


TT{t) - 7r(s)||'rF < 


— s 
a 



Clearly if e > 0, then having 


|t — s| <6 

implies ||vr(t) — 7r(s)||7’\/ < e. 


ea 


This shows that vr is continuous at s G [0,1]. Because one can do this for any 
s G [0,1], it is seen that vr is continuous with respect to the total variation 
norm on [0,1]. Because <5 does not depend on the value of s G [0,1], it is 
shown that tt is uniformly continuous. 
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